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1.  Introduction 


The  top-level  objective  of  this  SBIR  Phase  II  project  was  to  build  a  prototype  virtual  cockpit  that 
included  force  and  tactile  feedback.  We  achieved  this  top-level  objective  and  all  key  technical 
objectives  discussed  in  section  2  as  well.  We  discuss  more  of  each  of  the  technical  objectives 
and  our  approach  in  detail  later  in  the  report  when  we  present  our  system  concept  in  Section  3, 
and  Phase  II  results  in  Section  4. 

System  overview  and  project  accomplishment 

The  user  wears  a  head  mounted  display  that  presents  stereo  imagery  of  a  cockpit  interior, 
including  the  instrument  panel,  as  well  as  the  out-the-window  scenery.  A  representation  of  the 
user's  hand  is  also  rendered  in  the  scene.  The  user  may  actuate  a  variety  of  controls  on  the 
instrument  panel,  and  can  accurately  feel  the  forces  and  surface  textures  of  the  controls.  The 
simulator  can  be  reconfigured  entirely  in  software  to  represent  different  cockpits.  The  feel  of 
the  instrument  panel  controls  is  provided  by  a  servomechanism  device  that  places  actual 
physical  controls  in  their  correct  positions,  orientations,  and  configurations.  A  tracker  and  data 
glove  continually  provide  the  position  of  the  user's  hand  and  fingers  to  a  computer.  The 
computer  senses  the  position  as  the  person  reaches  for  a  control.  Using  the  extrapolated  data, 
the  computer  commands  the  servomechanism  system  to  place  the  correct  type  of  control  in  the 
correct  position  to  be  actuated.  The  servo  system  has  a  "touch  panel"  that  contains  examples  of 
a  dozen  or  so  different  t5rpes  of  controls,  such  as  toggle  switches,  knobs,  and  push  buttons,  that 
are  used  repeatedly  to  represent  any  number  of  instrument  panel  controls. 

The  system  is  called  a  TOPIT™  ^  Touched  Objects  Positioned  In  Time.  One  key  aspect  of 
the  system  is  building  a  servo  system  that  moves  fast  enough  to  always  have  the  control  in 
place  before  the  user's  hand  reaches  it.  Another  key  aspect  is  achieving  precise  low-latency 
tracking  of  both  the  user's  head  and  the  user's  hand.  The  tracking  must  be  accomplished  in  the 
presence  of  the  moving  metal  elements  and  the  electric  motors  of  the  servo  system;  a  hybrid 
magnetic/inertial  tracker  was  developed  to  meet  these  requirements.  The  system  has  three 
computers:  an  SGI  Onyx/RealityEngine2  that  does  the  imagery,  a  Pentium-based  PC  that  does 
the  tracking,  and  a  VME-based  servo  control  system. 

The  TOPIT  Force/Tactile  Feedback  System  concept  drawing  [Figure  1-1]  shows  the  proof- 
of-concept  demonstrator  being  used  to  simulate  an  aircraft  cockpit.  The  central  issue  of  the 
feasibility  of  the  scheme  is  establishing  and  meeting  the  timing  requirements  for  determining 
the  touched-object  and  moving  it  into  place  in  time. 

However,  while  basic  feasibility  was  established  in  Phase  I,  construction  of  a  demonstrator 
during  the  Phase  II  effort  required  the  careful  design  and  integration  of  mechanical,  electro¬ 
mechanical,  and  computer  controlled  devices  to  meet  project  objectives. 

Overall,  the  major  technical  challenges  were  met.  In  particular,  robotic  hardware  was  built 
to  position  the  controls  with  the  speed  and  accuracy  required,  and  a  sophisticated  tracker  and 
an  alternative  tracker  were  built  to  provide  the  accuracies  required  for  position  and 
extrapolation.  The  most  difficult  aspect  of  the  program  turned  out  to  be  getting  all  of  the  bugs 
out  of  the  complex  system  under  severe  budget  constraints.  In  this  last  respect  we  were  largely 
successful,  but  not  entirely.  The  main  limitations  of  the  final  prototype  lie  in  the  fine  points  of 
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getting  the  software  to  run  completely  smoothly  and  reliability.  We  view  none  of  the  present 
limitation  as  being  fundamental. 


Report  organization 

Section  1  presents  an  overview  of  the  project  and  snapshots  of  subsystems  and  components 
the  prototype  developed.  Section  2  discusses  the  technical  objectives  of  the  project.  Section  3 
discusses  the  system  concept  and  implementation,  and  section  4  compares  the  results  of  the 
phase  II  effort  to  the  objectives  and  the  original  designs  for  the  project.  Section  5  presents  the 
conclusions. 


Figure  1-1  TOPTT  concept.  Physical  switches  and  knobs  are  positioned  in  a  virtual 
environment  under  software  control  to  provide  flexible  force  and  tactile  feedback. 
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Figure  1-2  TOPIT  Prototype. 
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Figure  1-3.2  Joystick  and  instrumented  glove. 
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Figure  1-3.3  Multisensor. 


Figure  1-3.4  Throttle  and  emergency  stop  button. 
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Figure  1-3.5  Magnetic  tracker  transmitter. 


Figure  1-3.6  Y-axis  servo  motor. 
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Figure  1-3.7  Servo  electronics  cabinet. 


7 


Figure  1-3.8  Touchpanel. 
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2.  Summary  of  Technical  Objectives  and  Approach 

The  primary  objective  of  the  Phase  II  effort  was  to  design,  construct,  and  evaluate  the  TOPIT 
force  and  tactile  feedback  system  through  a  complete  implementation  of  a  virtual  cockpit.  We 
considered  developing  a  partial  implementation,  without  the  visual  simulation  of  the  virtual 
environment.  The  visual  environment,  however,  was  necessary  to  guide  the  user  to  each 
specific  point  in  virtual  space  where  a  virtual  control  was  located.  Without  the  visual 
simulation,  the  touchboard  could  only  be  guided  to  mirror  hand  and  finger  position,  and  the 
demonstration  would  miss  the  whole  aspect  of  predicting  hand  trajectory,  selecting  the  correct 
control,  and  fixing  the  control  position  in  time  to  be  touched.  Also  missing  would  have  been 
the  aspect  of  treating  head  tracker  and  image  generator  delays.  With  so  much  missing,  we 
concluded  that  a  partial  implementation  would  be  unconvincing  m  proving  the  TOPIT  concept. 
The  approach  we  adopted  paid  special  attention  to  the  risk  areas  identified  in  the  Phase  I 
study.  The  risk  areas,  identified  in  the  Phase  II  proposal,  and  our  approach  to  each  key  risk 
area  were  as  follows: 

(1)  We  wanted  to  build  a  positioning  system  that  moved  fast  enough,  but  without 
excessive  size,  power,  or  cost  was  to  be  approached  through  a  combination  of  rapid 
prototyping,  in  which  the  linear  transport  mechanism  for  the  x-axis  positioning  was  built 
experimentally  using  stepper  motor  and  servomechanism  implementations,  and  payload 
weight  was  minimized  through  careful  design  that  encompassed  the  use  of  lightweight 
materials. 

(2)  We  needed  to  ensure  the  tracking  system  provided  sufficient  accuracy  in  the 
presence  of  the  electromagnetic  noise  and  moving  metal  objects  of  the  positioning  system  was 
approached  by  use  of  a  pulsed  rather  than  continuous  wave  tracker,  synchronization  of  tracker 
pulses  between  motor  steps,  noise  minimization  by  shielding,  and  by  careful  tracker 
transmitter  placement.  If  problems  persisted,  a  noise  immune,  but  somewhat  encumbering, 
mechanical  tracker  was  to  be  used  to  support  development. 

(3)  We  needed  to  design  hand  motion  prediction  algorithms  that  predicted  which 
control  would  be  touched  while  sufficient  time  remained  to  put  it  in  place  was  first  approached 
at  the  system  level  using  the  basic  hand  motion  data  obtained  in  Phase  I.  These  data  bound  the 
performance  of  the  algorithm.  However,  considerable  experimentation  were  made  to  fine  tune 
the  algorithms.  Also,  an  alternative  tracking  system  was  developed  that  minimizes  the  need  for 
such  prediction  algorithms. 

(4)  Keeping  computation  and  control  lags  small  enough  so  that  the  positioning 
system  had  sufficient  time  to  position  the  touchboard  was  a  fundamental  systems  engineering 
task  required  careful  accounting  of  each  time  lag  in  the  system.  Continual  refinement  of  the 
timing  budget  allowed  early  identification  of  problems.  Computational  problems  could  be 
treated  by  using  dedicated  board  level  processors  for  the  control  algorithms,  by  microcoding 
key  computations,  and  by  using  interrupt-driven  synchronized  event  processing. 

(5)  Providing  redundant  safety  systems  to  protect  the  operator  during  development 
and  use  was  considered  to  employ  software  to  ensure  the  positioning  system  is  commanded  to 
stop  before  the  tracked  hand  moves  into  the  motion  space,  an  independent  light  curtain 
electronic  system  that  directly  shuts  down  the  system  upon  any  intrusion  into  the  motion 
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space,  and  mechanical  guards  around  the  working  mechanisms  to  ensure  than  intruding 
elements  were  deflected  rather  than  caught  or  pinched. 

The  identified  technical  risks  made  the  Phase  11  implementation  a  major  systems 
engineering  challenge.  Along  with  the  direct  risk  of  meeting  the  technical  objectives  was  the 
associated  risk  of  keeping  the  project  on  schedule  and  within  budget  as  the  various  challenges 
were  faced.  The  results  are  presented  in  the  following  two  chapters. 
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3.  System  Concept 


I V  — ^  ^  v"--  '  ^  ■»  '1  V  ' 


A  traditional  flight  simulator  is  built  using  a  replica  of  the  cockpit  of  the  aircraft  being 
simulated.  Building  a  replica  cockpit  is  expensive,  as  a  different  replica  cockpit  is  needed  for 
every  type  of  aircraft  to  be  simulated,  and  it  is  difficult  to  keep  up  with  changes  made  to  the 
real  aircraft.  Conceptually,  it  would  be  better  to  have  a  virtual  cockpit  in  which  the  elements  of 
the  cockpit  are  determined  entirely  by  software.  Then  the  expense  of  constructing  physical 
replicas  could  be  saved,  one  simulator  could  be  used  for  many  different  types  of  aircraft,  and 
after  the  simulators  are  in  service  the  simulators  could  be  quickly  updated  to  reflect 
modifications  in  the  real  aircraft. 

For  a  virtual  cockpit,  the  appearance  of  a  cockpit  can  be  represented  by  computer  generated 
imagery  on  a  head-mounted  display  (HMD)  worn  by  the  user.  The  fidelity  of  this  approach  is 
limited  by  the  resolution  of  the  HMD  and  by  the  realism  of  the  computer  generated  imagery 
for  the  display.  HMD  technology  and  image  generator  technology  are  such  that  the  best 
currently  available  technology  is  probably  barely  acceptable  for  the  application,  and  even  then 
at  relatively  high  cost.  However,  current  trends  toward  lower  cost  and  improved  performance 
should  close  the  performance  gap  considerably  within  a  few  years'  time. 

In  addition  to  a  visual  simulation,  a  virtual  cockpit  also  needs  a  simulation  of  the  force  and 
tactile  sensations  of  touching  the  controls.  The  controls  include  the  primary  controls  and  the 
instrument  panel  controls.  The  primary  controls  are  the  joystick  and  rudder  pedals  or  their 
equivalents  for  steering  the  aircraft.  The  instrument  panel  controls  include  switches,  knobs, 
push  buttons,  and  keypads.  Replica  controls  could  be  provided  to  be  used  with  the  simulated 
imagery,  but  doing  so  would  not  meet  the  objective  of  having  a  simulator  that  is  reconfigurable 
in  software. 

For  the  prototype  virtual  cockpit  discussed  here,  replicas  were  used  for  the  primary 
controls,  but  a  software  reconfigurable  approach  was  adopted  for  the  instrument  panel 
controls.  Because  the  simulator  user  is  wearing  a  head-movmted  display,  and  because  the  user 
touches  only  one  instrument  control  at  a  time,  it  suffices  to  present  to  the  user  only  the  single 
control  being  touched.  This  is  accomplished  by  using  a  collection  of  about  a  dozen  different 
tjrpes  of  physical  replicas  of  controls,  and  putting  the  correct  type  into  the  correct  place  to  be 
touched  whenever  the  user  actuates  a  control. 

To  select  the  correct  t5q)e  of  control  and  put  it  into  place,  the  user's  hand  and  fingers  must 
be  tracked  and  the  positions  extrapolated  forward  to  determine  which  control  will  be  grasped. 
A  robotic  mechanism  then  quickly  puts  the  correct  type  of  control  into  place  in  time  to  be 
actuated.  A  user  may  believe  that  different  toggle  switches  are  being  flipped  at  different  places 
on  the  instrument  panel,  but  in  fact  the  same  toggle  switch  is  being  touched  in  aU  the  different 
positions.  A  mechanism  must  be  provided  to  put  the  switch  in  the  correct  "up"  or  "down" 
position  while  the  switch  is  being  moved  to  a  new  position.  Similarly,  rotary  controls  must  be 
brought  into  correspondence  with  the  way  each  control  appears  in  the  user's  HMD  imagery. 

For  the  concept  to  be  practical,  the  few  replica  controls  must  be  moved  rapidly  to  stay 
ahead  of  the  user's  hand  motions.  The  requirements  were  quantified  by  analyzing  cockpit 
videotapes  taken  in  flight  and  also  videotapes  taken  in  a  lab  setup.  In  the  lab,  a  number  of  non¬ 
pilot  subjects  were  videotaped  as  they  actuated  switches  and  knobs  in  a  prescribed  sequence. 
Timing  requirements  were  determined  by  stepping  through  the  videotapes  frame-by-frame 
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and  recording  the  times  required  to  reach  the  controls.  The  derived  requirements  were  that  the 
controls  must  be  repositioned  with  an  acceleration  of  up  to  four  g's  and  a  speed  of  about  three 
meters  per  second.  Maximum  acceleration  and  deceleration  are  required  when  closely-spaced 
controls  are  actuated  in  sequence. 

3.1  System  Configuration 

The  system  is  designed  with  three  major  subsystems,  one  each  for  robotics,  tracking,  and  visual 
simulation  [Fig.  3.1-1].  Each  subsystem  is  controlled  by  its  own  computer,  with 
communications  links  transferring  data  among  the  three  control  computers. 


Figure  3.1-1  Three  major  subsystems. 

The  tracking  subsystem  is  built  around  a  personal  computer  running  the  QNX  real  time 
operating  system  [Figure  3.1-2].  The  tracking  computer  interfaces  with  the  hardware  that 
measures  the  position  and  orientation  of  the  user’s  head  and  right  hand  and  runs  software  that 
filters  and  extrapolates  the  tracking  data.  It  determines  which  switch  the  user  is  about  to 
actuate  and  sends  commands  to  the  robotics  subsystem  to  move  the  selected  switch  into  place. 
It  keeps  track  of  the  orientations  to  which  the  knobs  and  toggle  switches  are  moved.  It  also 
interfaces  to  the  user's  flight  control  joystick  and  throttle  and  computes  the  position  of  the 
simulated  aircraft.  The  tracking  computer  sends  the  positions  and  orientations  of  the  head, 
hand,  and  switches  to  the  visual  simulation  subsystem,  which  in  turn  generates  imagery  for 
viewing  in  the  user's  HMD. 

The  robotics  subsystem  includes  a  VME-rack  with  a  control  processor  and  interfaces,  servo 
power  supplies  and  amplifiers,  and  power  distribution  circuitry.  The  VME-based  control 
processor  receives  high  level  commands  from  the  tracking  computer  over  a  38.4  Kb  serial 
interface.  The  commands  from  the  tracking  computer  instruct  the  robotics  subsystem  to  move 
each  of  the  servo-driven  positioning  mechanisms  to  prescribed  locations  or  orientations.  The 
robotics  control  processor  carries  out  the  commands  by  generating  control  voltages  for  each  of 
the  servo-motor  amplifiers.  The  motors  are  equipped  with  digital  shaft  encoders  and  each 
motor  channel  is  run  closed-loop  with  an  update  rate  of  approximately  100  Hz.  Each  channel  is 
tuned  for  the  inertia  and  spring  constants  associated  with  the  channels'  hardware. 


12 


Figure  3.1-2  TOPIT  tracking  computer,  magnetic  tracker  electronics  (right)  and  HMD 

electronics  (on  top  of  computer). 

The  visual  subsystem  is  built  around  a  Silicon  Graphics  Onyx  computer  having  a 
RealityEngineZ  image  generator.  The  visual  computer  receives  data  from  the  tracking 
subsystem  over  a  dedicated  Ethernet  link  having  less  than  one  millisecond  latency.  The  visual 
computer  has  a  database  of  polygons  modeling  the  cockpit  interior,  the  user's  hand,  and  terrain 
outside  the  simulated  aircraft.  It  assembles  the  scene  from  the  polygon  models,  putting  each 
model  in  its  correct  relative  position.  A  dataglove  worn  by  the  user  provides  the  positions  of 
the  fingers  directly  to  the  visual  computer. 

3.2  Tracker 

Magnetic  trackers  are  commonly  used  in  virtual  reality  systems.  They  use  compact,  lightweight 
sensors,  are  imencumbering,  measure  all  three  position  coordinates  and  all  three  orientation 
angles,  and  are  economical.  The  limitations  of  magnetic  sensors  are  that  metallic  objects  distort 
the  tracker  fields  thereby  producing  static  errors,  they  are  susceptible  to  interference  from 
electrical  noise  sources,  and  there  tends  to  be  lags  in  the  measurements.  The  lags  come  from 
filtering  the  noise  inherent  in  the  measurements.  In  many  applications,  none  of  the  limitations 
prove  severe.  For  the  virtual  cockpit,  however,  the  tracking  could  not  lag  significantly  and 
must  work  in  the  presence  of  the  metal  and  motors  of  the  robotic  positioning  device. 

One  alternative  to  magnetic  tracking  was  mechanical  tracking.  A  mechanical  tracker  uses 
stiff  rods  connected  by  joints  having  encoders.  Mechanical  trackers  are  low  cost,  extremely 
accurate,  immune  to  noise,  and  have  no  appreciable  lag.  Unfortunately,  mechanical  trackers  are 
encumbering  since  they  require  a  mechanical  linkage  to  the  users  head  or  hand.  They  are  best 
used  when  the  space  of  possible  motion  is  small,  and  might  be  acceptable  for  head  tracking  a 
seated  user.  For  hand  tracking  in  a  virtual  cockpit,  the  encumbrance  would  not  be  acceptable  in 
the  long  run.  Nonetheless,  mechanical  tracking  could  be  a  backup  method,  at  least  for  lab 
evaluation  of  the  virtual  cockpit. 
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There  were  a  number  of  optical  tracking  systems  available.  These  systems  use  a  variety  of 
principles  for  tracking.  Some  use  high  resolution  cameras  tracking  reflective  markers.  Others 
use  sensors  that  detect  a  scanning  infrared  laser.  Optical  tracking  systems  are  typically  so 
accurate  that  the  orientation  of  a  surface  can  be  computed  by  tracking  three  points  on  the 
surface.  Optical  tracking  would  be  a  good  choice  for  a  virtual  cockpit,  but  the  cost  of 
commercially  available  systems  ruled  it  out  for  the  prototype. 

The  alternatives  were  to  work  with  the  limitations  of  magnetic  trackers  or  to  attempt 
development  of  a  low-cost  optically-based  tracking  system.  We  opted  to  work  with  the 
magnetic  tracker.  To  minimize  magnetic  field  distortions,  the  robotic  mechanism  would  have  to 
be  made  from  non-magnetic  material.  Aluminum  was  tested  and  foxmd  to  be  nearly  as  bad  as 
carbon  steel  m  inducing  tracker  distortions;  it  apparently  induced  distortions  in  the  electric 
field  component  of  the  tracker  transmission.  The  best  metal  was  non-magnetic  stainless  steel 
(series  300),  so  that  was  preferred  for  construction.  Wood  or  plastic  might  have  been  used,  but 
the  structure  could  not  be  made  acceptably  stiff. 

As  it  turned  out,  the  distortions  due  to  the  metal  structure  were  up  to  about  4  cm  of  error, 
which  could  be  reduced  substantially  by  calibration  and  look-up  tables.  The  goal  was  to 
provide  overall  tracking  accuracy  of  about  5  mm,  which  seems  achievable. 

To  treat  the  problem  of  tracker  lag,  an  inertial  sensor  package  was  added  to  the  magnetic 
hand  tracker.  The  package  initially  consisted  of  three  miniature  accelerometers  and  three 
angular  rate  sensors.  This  inertial  package  was  larger  than  desired,  about  three  inches  square 
and  an  inch  think;  however,  it  could  be  mounted  on  the  forearm  rather  than  on  the  hand  itself. 

The  alignment  of  the  axis  of  each  sensor  was  required  to  be  orthogonal  in  order  for  the 
software  to  receive  correct  information.  This  was  not  attainable  with  the  aforementioned  setup, 
so  two  replacement  sensors  were  purchased  -  a  triaxial  rate  gyro  and  a  triaxial  accelerometer. 
This  new  inertial  package  was  slightly  more  compact  and  could  be  fitted  on  the  user's  wrist. 

Combined  with  inertial  data,  the  magnetic  tracker  data  could  be  smoothed  with  only  about 
a  fifth  of  its  typical  lag,  roughly  30  milliseconds  rather  than  150  milliseconds.  Also,  the  accurate 
velocity  and  acceleration  measurements  enabled  better  extrapolation  of  the  hand  position. 
Extrapolation  is  required  to  compensate  for  delays  of  30  to  60  miUiseconds  in  the  image 
generator,  and  to  extrapolate  the  hand  position  to  determine  which  switch  is  selected. 

The  magnetic  and  inertial  tracking  data  are  combined  in  software  using  Kalman  filtering,  a 
technique  often  applied  in  multi-sensor  navigation  systems.  The  computational  requirements 
of  the  filter  are  just  within  the  capabilities  of  a  200  MHz  personal  computer,  although  they 
could  be  reduced  with  more  optimization. 

3.3  Robotic  Mechanism 

The  starting  point  for  selecting  a  robotic  mechanism  was  to  consider  off-the-shelf  devices 
such  as  industrial  robots.  The  robot  must  position  a  payload  having  an  assortment  of  controls 
together  with  the  motors  necessary  to  reposition  the  rotary  controls  and  toggle  switches.  An 
initial  estimate  was  that  the  payload  would  weigh  about  five  kilograms,  although  the  ultimate 
design  totaled  about  eight  kilograms  —  a  consequence  of  the  stainless  steel  construction. 

Industrial  robots  were  available  which  meet  the  requirements,  but  they  are  large,  high 
powered,  and  expensive.  Industrial  robots  are  designed  to  have  a  long  reach  into  a  large 
workspace,  and  consequently  are  built  with  heavy  links  which  in  turn  must  be  driven  by 
powerful  drive  mechanisms.  Cockpit  instrument  panels  are  wide  and  fairly  high,  but  the  panel 
surface  does  not  encompass  much  depth.  A  custom  robotic  device  was  designed  to  take 
advantage  of  the  restricted  workspace.  It  cost  less  and  is  safer  than  an  industrial  robot. 
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The  large  reach  of  the  industrial  robot  would  have  posed  a  safety  problem.  Potentially,  the 
robot  could  move  respectable  masses  at  high  speeds  into  the  space  of  the  user.  Since  it  would 
not  be  acceptable  to  operate  only  with  software  limits  the  robot  would  have  to  be  physically 
modified  to  make  it  impossible  to  travel  into  the  user's  space.  The  customization  required 
would  further  added  to  ^e  cost  of  the  device. 

Finally,  industrial  robots  are  not  t)q)ically  made  of  non-magnetic  stainless  steel.  Making  a 
new  device  permitted  constraining  the  design  to  be  compatible  with  magnetic  tracking.  In  a 
new  design,  the  electric  motors  could  be  positioned  as  far  away  from  the  trackers  as  possible. 

The  manipulator  design  recalls  some  of  the  design  features  of  an  old-fashioned  pen  plotter 

[Figures  3.3-1  and  3.3-2].  The  horizontal  and  vertical  axes  are  driven  by  Kevlar^M  cables.  Using 
cables  for  both  drive  mechanisms  avoids  making  the  outer  axis  motor  bear  the  burden  of 
having  to  move  the  inner  axis  motors.  Both  major  axis  drive  motors  are  affixed  to  the  frame, 
one  on  either  side,  near  the  groimd,  and  back  from  the  trackers.  A  relatively  small  motor, 
which  moves  the  payload  in  and  out,  is  carried  with  the  payload.  The  electronics  cabinet, 
which  houses  the  servo  electronics  and  system  power  control  and  safety  circuitry,  can  be  seen 
to  the  left  of  the  user  [Figure  3.3-2]. 


Figure  3.3-1  TOPIT  system  showing  operator's  station,  X-Y  manipulator,  and  touchboard. 
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Figure  3.3-2  The  final  design  uses  cables  to  position  the  switch  payload  in  x  and  y  axes. 

The  main  design  feature  for  safety  is  constraining  the  user  and  the  robotic  mechanisms  to 
their  own  workspaces.  The  user  must  cross  into  the  robot's  workspace  to  touch  the  payload 
controls,  but  the  hand  is  tracked  and  the  software  is  designed  to  bring  the  mechanism  to  a  halt 
before  the  hand  crosses  into  the  mechanism  area.  Still,  one  must  account  for  possible  software 
failures,  for  untracked  parts  of  the  user’s  body,  and  for  bystanders.  These  additional  safety 
provisions  are  discussed  in  section  4.5. 

In  the  current  design  only  the  head  and  the  right  hand  are  tracked.  Tracking  the  left  hand  is 
mainly  a  cost  issue,  and  doing  so  would  allow  controls  to  be  actuated  with  either  hand  as  well 
as  enhancing  safety.  The  imtracked  left  hand  is  required  to  be  kept  on  the  throttle.  A  switch  on 
the  throttle  must  be  continually  depressed;  if  it  is  released  the  mechanism  halts.  The  throttle 
switch  tends  to  keep  the  user  properly  seated  away  from  the  mechanism.  A  second  switch 
could  be  added  to  the  seat  back  to  further  ensure  the  head  is  kept  back  from  the  mechanism; 
leaning  forward  would  release  the  seat  switch  and  stop  the  mechanism. 

The  payload  [Fig.  3.3-3]  moves  with  maximum  speed  about  equal  to  a  hand  moved 
laterally  to  activate  a  switch.  This  is  not  fast  enough  to  cause  a  serious  injury  if,  due  to  a  system 
failure,  it  were  to  hit  the  user's  hand  in  motion.  A  potential  danger  lies  in  pinch  points,  where 
the  users  hand  might  be  caught  in  a  closing  space  between  the  frame  and  the  payload  or 
traveler.  Pinch  points  are  prevented  by  making  the  frame  oversized  and  movmting  rubber 
blocks  to  stop  mechanical  travel  short  of  the  frame. 

An  emergency  stop  circuit  is  included  in  the  design.  This  circuit  is  hardwired  to  a  single 
relay  that  disconnects  and  then  short-circuits  the  drive  motors,  quickly  bringing  the 
mechanism  to  a  halt.  When  the  virtual  cockpit  is  in  operation,  an  observer  can  actuate  one  of 
two  emergency  stop  switches  if  the  user  or  a  spectator  gets  too  close  to  the  mechanism. 


Figure  33-3  The  payload  includes  switches  and  knobs  which  are  rotated  to  the  needed 

orientation. 

Covers  would  be  added  to  any  production  device  to  prevent  a  bystander  from  reaching  any 
of  the  drive  mechanisms  from  the  sides  or  rear  of  the  device. 

3.4  Visual  Simulation 

The  Onyx  RealityEngine2  computer  uses  position  data  from  the  tracker  to  prepare  the  visual 
scene  from  pre-stored  polygon  models  of  the  cockpit,  the  user's  hand,  and  the  out-the-window 
terrain  [Fig.  3.4-1].  The  Onyx  computer  runs  a  real  time  version  of  UNIX  in  two  processors,  and 
we  wrote  ttie  visual  simulation  using  Silicon  Graphic's  Performer  application  package. 

There  is  a  delay  of  one  to  two  video  frames  in  generating  the  image,  marked  from  the  time 
position  data  arrives  in  the  tracking  computer  until  the  generated  image  is  displayed  to  the 
user.  The  image  is  generated  to  correspond  to  where  each  moving  element  of  the  scene  is 
expected  to  be  at  the  time  when  the  image  appears.  Consequently,  the  position  and  orientation 
of  every  moving  element  in  the  scene  must  be  extrapolated  forward  from  the  time  at  which  the 
position  and  orientation  of  the  element  were  measured  to  the  time  at  which  the  image  appears. 
Simple  extrapolation  using  velocities  and  accelerations  works  adequately  for  times  up  to  about 
100  milliseconds. 

The  imagery  is  presented  to  the  user  on  a  head-mounted  display.  Separate  images  are 
computed  for  each  eye  to  provide  true  stereo.  The  user's  judgment  of  his  hand  position  relative 
to  the  instrument  panel  is  helped  significantly  by  having  stereo  imagery. 


Figure  3.4-1  Generated  imagery  includes  the  user's  hand. 

A  moderately  priced  liquid-crystal  based  head-mounted  display  is  used,  the  Virtual 
Research  VR-4.  This  HMD  provides  a  resolution  of  about  320  x  480  pixels.  This  resolution  is 
adequate  to  see  and  grasp  the  controls,  but  it  is  not  sufficient  to  read  either  normal-sized 
control  labels  or  many  instrument  panel  displays.  The  compromise  in  resolution  was  forced  by 
the  economics  of  the  protot5q)e.  High  resolution  head-mounted  displays  are  expensive,  and  the 
increased  resolution  requires  more  expensive  image  generation  capability.  In  the  development 
system,  the  emphasis  was  on  demonstrating  the  feasibility  of  the  force  and  tactile  feedback 
mechanism  rather  than  the  display. 

The  polygon  processing  capacity  of  the  image  generator  (about  220K  polygons  per  second) 
limits  the  scene  complexity.  The  use  of  stereo  imagery  cuts  the  scene  complexity  in  half  relative 
to  what  it  would  be  otherwise.  The  cockpit  interior  is  inherently  complex,  with  the  knobs  and 
switches  modeled  in  three  dimensions.  The  desired  frame  rate  is  at  least  30  frames  per  second, 
and  the  allowable  polygon  complexity  per  frame  is  lowered  in  proportion  to  the  frame  rate. 

The  technology  of  HMD's  and  image  generators  is  advancing  rapidly,  so  that  these  system 
elements  are  expected  to  be  less  of  a  limiting  factor  in  the  future.  The  possibilities  of  advancing 
technology  was  part  of  the  reason  for  partitioning  the  tracking  and  visual  simulation 
subsystems  around  separate  computers.  The  partitioning  was  designed  to  simplify  substitution 
of  lower-cost  image  generation  technology  without  having  to  recode  the  PC-based  tracking 
computer  software.  The  partitioning  could  also  help  in  making  the  hybrid  magnetic  and 
inertial  tracker  with  its  Kalman  filter  into  a  separate  PC-based  subsystem  for  other  applications. 


3.5  System  Integration 

The  system  as  described  is  currently  working  well  enough  to  demonstrate  the  feasibility  of  the 
concept,  although  there  are  improvements  to  be  made.  A  few  of  the  lessons  learned  in  system 
integration  can  be  cited. 

Initially,  the  software  was  designed  so  that  the  payload  would  be  moved  to  mirror  the 
position  the  hand  until  the  hand  was  within  a  few  inches  of  the  surface  of  the  virtual 
instrument  panel.  When  the  hand  reached  the  pre-determined  close  distance,  the  software 
would  pick  the  switch  to  be  grasped,  put  the  switch  into  place,  and  freeze  the  payload  position 
until  the  switch  was  actuated  and  the  hand  withdrawn. 

The  error  in  this  design  soon  became  apparent.  The  robotic  system  is  designed  to  accelerate 
the  payload  at  up  to  4  Gs.  When  the  hand  was  still  distant  from  the  panel,  the  attempt  to 
mirror  the  position  of  the  hand  with  the  payload  produced  a  great  deal  of  pointless  violent 
motion  of  the  payload.  The  cure  was  to  put  an  extra  filter  on  the  position  data  given  to  the 
robotic  subsystem.  The  extra  filter  has  a  time  constant  which  is  adjusted  to  provide  smoother 
position  following  when  the  hand  is  further  from  the  virtual  instrument  panel. 

Another  unanticipated  problem  was  resonance  in  the  mechanical  frame  of  the  positioning 
mechanism.  The  total  weight  of  the  traveler  mechanism  and  the  payload  is  approximately  44 
pounds.  When  accelerated  at  4  Gs,  the  resulting  reaction  force  is  therefore  about  176  pounds. 
The  frame  is  welded  from  2  inch  square  stainless  steel  tubing  and  is  very  stiff. 

However,  tracking  movements  of  the  hand  produces  frequencies  which  excite  resonances 
in  the  structure.  When  resonating,  the  deflections  at  the  comer  of  the  structure  may  be  as  much 
as  a  centimeter.  It  is  not  clear  if  the  deflections  actually  degrade  system  performance,  but  the 
fear  is  that  they  affect  the  servo  control  loops.  The  shaking  is  also  disturbing  to  bystanders. 
Custom  pampers  were  added  across  the  diagonals  of  the  structure  to  dissipate  the  resonant 
energy. 

A  shell,  floor,  and  an  integrated  pilot's  seat  were  added  to  the  frame  for  aesthetic  reasons. 
A  compartment  imdemeath  the  pilot's  seat  was  also  built  to  house  the  sensor  electronics. 

3.6  Future  VR  Systems/Phase  III  Applications 

Forethought  in  system  partitioning  and  timing  analysis,  as  well  as  making  tradeoffs  among  the 
limitations  of  subsystems  are  central  to  good  virtual  reality  systems  design.  Systems  with 
human/robotic  interaction  inevitably  pose  serious  safety  considerations.  With  present 
technology  and  experience,  it  is  feasible  to  build  a  limited  class  of  virtual  reality  systems,  such 
as  the  virtual  cockpit,  to  provide  force  and  tactile  feedback. 

Robotic  positioning  systems 

The  concept  of  robotic  positioning  of  touched  objects  is  not  a  universal  solution  to  the  problem 
of  providing  force  and  tactile  feedback.  It  applies  when  there  is  a  limited  class  of  objects  to  be 
touched,  when  fidelity  is  important,  when  the  simulation  of  external  forces  is  important,  and 
when  safety  constraints  can  be  met.  Alternative  methods  include  special  gloves  having  air 
bladders  or  other  touch  stimulating  transducers,  exoskeleton  devices  attached  to  the  hand  or 
body,  and  robotic  devices  continuously  attached  to  specialized  tools  for  simulating  the  forces 
encountered  in  the  tool  use. 

Within  its  realm,  the  method  of  positioned  objects  does  offer  interesting  possibilities.  Note 
that  in  the  virtual  cockpit,  the  system  could  easily  provide  the  capability  for  touching  the 
window  glass  or  the  flat  surfaces  of  the  cockpit  surfaces.  In  an  architectural  walk-through  or 
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entertainment  system,  the  user  might  touch  various  surfaces  of  the  environment,  presented  by 
a  robot  having  a  selection  of  surfaces.  The  performance  requirements  for  the  robotic 
mechanism  in  a  walk-through  environment  are  different  from  the  virtual  cockpit.  The  larger 
workspace  might  dictate  something  closer  to  an  industrial  robot  design,  but  larger  surfaces  and 
slower  user  motion  could  relax  the  speed  requirements  and  make  the  system  safe. 

A  robotic  system  could  also  initiate  contact.  Consider  a  virtual  reality  entertainment  system 
with  a  "dungeons  and  dragons"  scenario.  A  user  wearing  a  head-motmted  display  explores 
dark  passageways.  At  a  critical  juncture,  the  user  hears  a  sound  from  behind  and  at  the  same 
time  a  robotic  device  reaches  out  with  a  rubber  finger  to  deliver  a  poke  in  the  ribs.  There  is 
potential  for  such  systems. 

Hybrid  tracker 

The  hybrid  filter  could  be  commercialized  by  itself.  It  would  need  to  be  smaller  and  lighter 
weight.  The  hybrid  sensor  module  uses  two  types  of  sensors;  accelerometers  and  angular  rate 
sensors.  The  accelerometers  available  today  seem  suitable  but  the  angular  rate  sensors  are 
bulky  and  relatively  heavy.  For  the  hybrid  tracker  to  be  smaller  and  lighter  alternative  angular 
rate  sensors  are  needed.  Tliere  are  several  sensor  companies  already  working  on  better  angular 
rate  sensors  but  suitable  products  are  not  currently  available  -  perhaps  in  a  year  or  so. 

A  commercialized  hybrid  tracker,  along  with  the  Kalman  filter  software  we  developed, 
would  make  a  great  add-on  to  commercially  available  magnetic  trackers  produced  by 
Ascension  and  Polhemus  by  providing  an  accurate  low  lag  tracking  system.  Many  aerospace 
and  commercial  applications  would  benefit  from  such  enhanced  performance. 
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4.  Phase  II  Results 


W  have  accomplished  all  of  our  objectives.  A  picture  of  the  final  system  and  snapshots  of 
subsystems  and  components  is  presented  in  section  1.  Detailed  evolution  results  of  each 
subsystem,  compared  against  the  objectives,  are  described  in  the  following. 

4.1  Positioning  System 


Project  Objective  #1:  Build  a  positioning  system  that  moved  fast  enough  but  without  excessive 
size,  power,  or  cost. 


The  positioning  system  includes  the  X,  Y,  and  Z  axes  of  motion.  To  move  fast  enough  to  keep 
up  with  anticipated  user  motions,  the  TOPIT™  had  to  accelerate  at  4  Gs  and  move  at  100  inches 
per  second  in  both  the  X  and  Y  directions.  The  rates  were  achieved  in  both  axes  even  though 
the  weight  of  the  payload  exceeded  expectations.  Two  oversights,  a  miscalculation  of  the  drive 
drum  diameter  and  the  effects  due  to  gravity,  previously  prevented  the  Y  axis  from  attaining 
this  specification.  The  incorrectly  sized  drive  drum  was  corrected  by  designing,  fabricating,  and 
installing  a  larger  diameter  drive  drum.  The  effects  due  to  gravity  were  compensated  by  using 
a  coil  spring  attached  by  one  end  to  the  frame  and  the  other  end  around  the  new  drive  drum. 

A  significant  technical  achievement  in  the  X,  Y  motion  control  was  the  successful 
implementation  of  motion  control  software  which  not  only  controls  the  servos  in  such  a  way 
that  the  payload  is  transported  to  its  commanded  location  in  minimum  time  without  overshoot 
but  also  coordinates  X  and  Y  motions  such  that  straight-line  X-Y  motions  are  achieved. 
Straight-line  motions  minimize  the  demand  on  the  servo  power  supply  by  reducing  the 
commanded  acceleration  and  speed  on  the  axis  with  the  shortest  distance  to  travel  such  that 
the  X  and  Y  motions  take  the  same  amount  of  time  to  complete. 

We  incorporated  a  variable  gain  filter  on  X/Y  motions  so  that  the  servo  system  responds  to 
hand  motions  more  slowly  when  the  user’s  hand  is  more  than  a  few  inches  from  the  payload. 
The  filter  is  progressive;  the  further  away  the  user's  hand,  the  slower  the  response.  Highest 
performance  moves  (high  speed  and  high  acceleration)  are  only  needed  for  final  payload 
positioning  when  the  user's  hand  is  approaching  the  payload.  General  tracking  is  all  that  is 
required  otherwise. 

To  achieve  X  and  Y  drive  performance  and  minimize  the  required  power  and  cost  it  was 
necessary  to  minimize  the  weight  which  had  to  be  moved.  The  Z  drive  motion  is  attached  to 
and  moves  with  the  payload  by  the  X  and  Y  drive  motors.  In  an  effort  to  save  weight  and 
achieve  desired  performance  in  the  X  and  Y  directions  a  motor  was  chosen  for  Z  drive.  Initially, 
we  had  some  mechanical  difficulty  in  setting  up  the  Z  drive  correctly.  After  a  redesign  of  the 
linear  bearing  and  ball  screw  assembly,  the  Z  drive  now  works  weU;  although,  the  maximum 
travel  is  limited  to  four  inches  rather  than  the  specified  six  inches. 

Since  we  plaimed  to  build  a  virtual  aircraft  cockpit  the  overall  size  of  the  TOPIT 
manipulator  frame  was  dictated  largely  by  the  application.  One  design  goal  of  the  project  was 
to  have  the  ability  to  simulate  an  aircraft  cockpit  dashboard  area  42  inches  wide  by  30  inches 
high.  We  achieved  horizontal  (X  direction)  excursions  of  42  inches  but  safety  concerns  and  the 
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need  to  allow  emergency  stopping  areas  above  and  below  the  payload  limited  vertical 
excursions  to  about  22  inches  -  still  sufficient  for  proof-of-concept  testing.  We  believe  the 
overall  size  of  the  device  could  be  made  somewhat  smaller  by  repositioning  tumbuckles  used 
to  tension  the  X  and  Y  drive  cords  and  by  repositioning  several  of  the  pulleys  used  by  those 
cords.  A  somewhat  different  payload  and  touAboard  design  where  the  touchboard  (the  switch 
panel  on  which  the  user  controls  are  located)  is  partially  cantilevered  would  increase  the 
vertical  excursion  without  compromising  safety. 

The  following  subsections  describe  the  evolution  of  the  system.  While  the  various  efforts 
are  discussed  sequentially,  the  reader  should  note  there  was  considerable  overlap  in  these 
efforts. 

4. 1. 1  Desktop  prototype  hardware  development  and  testing 

To  confirm  our  motor  calculations,  verify  our  bearing  choice,  and  to  provide  a  means  of  testing 
the  magnetic  tracker  compatibility  we  decided  to  build  a  single  axis  desktop  protot5rpe  on 
which  performance  with  loads  of  up  to  25  pounds  were  tested.  The  prototype  used  a  custom 
workbench-like  structure  that  was  constructed  of  wood  so  that  it  would  not  produce  magnetic 
interference.  Initially  the  tracks  for  the  wheels  were  made  of  steel.  We  also  tried  aluminum  and 
stainless  steel  tracks  later.  Testing  suggested  we  needed  to  use  non-magnetic  (series  300) 
stainless  steel  for  the  proof-of-concept  system.  The  load  was  moved  using  low-stretch  Kevlar 
cords  and  a  series  of  pulleys  cormected  to  a  drum  which  in  turn  was  direct-coupled  to  a  two- 
horsepower  servo  motor. 

We  originally  planned  to  use  v-groove  wheels  and  track.  The  quality  of  the  wheel  bearings 
and  the  wheels  themselves  turned  out  not  to  be  satisfactory  so  we  used  cam  followers,  which 
have  heavy  duty  roller  bearings,  for  the  wheels  and  aluminum  chaimel  for  the  track  [Figure 
4.1-1].  Note  the  hxmbuckles  which  are  used  to  tension  the  Kevlar  cords. 


Figure  4.1-1  Desktop  prototype  with  cam  followers  used  for  wheels  and  aluminum  channel 

used  for  the  track. 

The  drive  mechanism  [Figure  4.1-2]  was  located  tmdemeath  the  desktop.  Note  the  hand 
crank,  which  was  used  for  initial  testing,  the  drive  drum,  and  the  Kevlar  cord  wrapped  around 
the  drum. 
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Figure  4.1-2  The  desktop  prototype  drive  mechanism. 


After  receiving  all  the  necessary  components  and  completing  construction  of  the  desktop 
prototype  it  was  wired  to  the  servo  electronics  and  testing  began.  The  safety  system  was 
verified  to  be  working  correctly.  The  homing  sequence  was  programmed  and  verified;  the 
homing  sequence  is  the  process  by  which  positioning  is  automatically  calibrated  by  moving  the 
payload  so  as  to  actuate  switches  at  each  end  of  the  device.  We  also  determined  the  sliding 
carriage  could  be  positioned  with  an  accuracy  of  1  mm  or  better  over  its  travel  -  more  accurate 
than  needed. 

The  ultimate  requirement  for  the  system  is  to  provide  4  Gs  of  acceleration  and  100  inches 
per  second  velocity  with  a  payload  of  25  pounds.  A  piece  of  scrap  iron  was  used  as  the  load  on 
the  desktop  prototype  [Figure  4.1-3]. 


Figure  4.1-3  Desktop  prototype  with  scrap  metal  load. 


The  test  envelope  was  first  expanded  to  1  G  of  acceleration  and  100  inches  per  second 
velocity  without  additional  payload  weight.  As  testing  progressed  we  identified  and  cured  the 
various  problems  that  surfaced.  Many  of  these  problems  would  have  occurred  later  on  the 
proof-of-concept  system  had  we  not  first  discovered  them  on  the  desktop  prototype.  We 
therefore  believe  the  desktop  prototype  effort  to  be  worthwhile  since  it  uncovered  problem 


23 


early  in  the  program  allowing  more  time  to  correct  them  before  construction  of  the  proof-of- 
concept  device.  Several  examples  of  problems  encountered  follow. 

First,  there  was  more  friction  in  the  unloaded  transport  mechanism  than  we  expected.  We 
changed  the  cable  pulleys  to  a  larger  type  with  better  bearings  and  that  helped,  but  did  not 
reduce  the  friction  to  our  expectations.  The  motor  is  powerful  enough  to  easily  overcome  the 
transport  friction,  but  we  still  wanted  to  reduce  it  further  and  experimented  accordingly. 

Second,  servo  power  amplifier  shut  itself  down  short  of  providing  all  the  power  we  needed 
to  achieve  4  Gs  of  acceleration.  We  decided  the  protection  limits  were  set  too  low,  so  we 
expanded  them  without  risking  the  amplifier.  We  later  upgraded  the  amplifier  to  allow  it  to 
handle  greater  loads. 

Third,  there  was  some  concern  with  the  motor  starting  to  heat  up  during  continuous 
operation.  This  turned  out  not  to  be  a  problem. 

We  were  concerned  that  Kevlar  cable  used  in  the  drive  transport  might  creep  or  be  too  stiff 
but  later  concluded  the  Kevlar  cable  worked  well. 

We  observed  motion  oscillation  before  settling  at  high  loads.  This  was  cured  by  correctly 
tuning  the  motion  control  software  to  account  for  the  "as  built"  mechanical  stiffness  of  each  axis 
of  motion.  Such  tuning  is  typical  of  a  servo  systems  and  Delta  Tau  (manufacturer  of  the  PMAC 
motion  controller  card)  provides  software  specifically  designed  to  permit  such  tuning. 

Test  objectives  for  the  desktop  protot3^e  were  established  and  software  written  to  support 
the  testing  needed  to  accomplish  the  objectives.  Desktop  prototype  test  objectives  and  test 
results  are  discussed  below: 


1)  Verifying  operation  of  the  limit-switch  safety  system 
The  limit  switch  system  worked  reliably  and  correctly. 

2)  Verifying  the  homing  and  position  calibration  sequence 

The  homing  and  position  calibration  software  worked  reliably  and  correctly. 

3)  Identifying  sources  of  friction  and  loading  in  the  transport  mechanism 

Excess  friction  was  traced  to  the  bearings  in  the  cable  pulleys.  Higher  performance  pulleys 
were  installed,  and  this  significantly  reduced  the  friction. 


4)  Establishing  ways  of  checking  and  maintaining  component  alignment 

A  four-tumbuckle  system  was  installed  and  has  proved  suitable  for  alignment.  However,  the 
approximately  six  inch  length  of  each  tumbuckle  reduced  the  usable  active  area  of  the  desktop 
prototype  by  almost  a  foot.  A  more  compact  tumbuckle  mechanism,  where  the  mechanism 
overlaps  wi^  the  width  of  the  payload  carrier,  was  designed  and  installed.  It  worked  fine  and 
provided  about  twelve  additional  inches  of  payload  travel. 


5)  Checking  the  positioning  accuracy  and  repeatability 

The  positioning  accuracy  is  better  than  a  millimeter,  and  we  had  no  problems  with  the  cable 
stretching  or  slipping  on  the  drum.  We  need  only  about  3-5  millimeters  accuracy. 
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6)  Establishing  the  acceleration  and  velocity  limits  of  4  Gs  and  100  inches/second 

After  a  few  problems  with  amplifier  shut  down  and  amplifier  failures  we  achieved  the  design 
goal  of  100  inches  per  second  for  large  excursions  of  the  manipulator.  We  also  set  a  second 
design  goal  of  4  Gs  for  small  excursions.  Both  of  these  goals  were  met  with  a  22  pound  load  - 
nominally  the  load  to  be  carried  by  the  "X"  axis  (horizontal). 


7)  Checking  for  design  limits,  such  as  potential  motor  heating,  in  continuous  operation 

The  payload  was  cycled  for  several  minutes  at  4  Gs  acceleration  -  a  time  duration  far  in  excess 
of  normal  operation.  Motor  heating  was  not  a  problem  but  the  extended  duration  of  maximum 
acceleration  testing  caused  one  of  the  amplifier  IGBTs  to  fail.  The  amplifier  has  subsequently 
been  upgraded  and  we  have  not  had  any  other  failures. 


8)  Rechecking  all  of  the  design  parameters  with  half  and  fidl  payload  weights 
Rechecking  the  design  parameters  with  half  and  full  loads  was  completed. 


In  the  process  of  exercising  the  desktop  prototype  several  other  concerns  arose;  selection  of 
a  slider  material  suitable  for  the  X  and/ or  Y  axes,  audible  system  noise,  and  Kevlar  drive  cord 
stretch.  Our  efforts  in  these  areas  are  discussed  below. 

We  tested  several  types  of  slider  material.  We  were  looking  for  a  durable  low  friction 
material.  We  found  that  Teflon-loaded  Delrin  sliding  against  a  stainless  steel  track  worked 
reasonably  well.  The  concern  was  that  the  plastic  would  gall,  which  is  what  happened  when 
we  tried  nylon  against  aluminum.  However,  the  Delrin  did  not  gall  nor  show  signs  of  wear  in 
our  tests,  even  though  the  stainless  steel  track  material  used  for  the  tests  was  not  finished  as 
smoothly  as  one  would  like.  The  final  product  would  have  a  smooth  finished  track.  Based  on 
the  success  of  these  tests,  we  used  the  Delrin/ stainless  steel  combination  for  the  y-axis  sUder  in 
the  final  design  for  the  proof-of-concept  demonstrator. 

We  noticed  the  system  was  noisier  than  we  would  prefer  when  the  payload  moved.  We 
thought  this  noise  might  be  distracting  to  a  user  even  if  the  user  was  wearing  a  headset.  We 
determined  the  noise  was  due  to  the  steel  payload  wheels  riding  on  the  aluminum  channel  we 
were  using  as  a  guide. 

The  moving  mechanism  was  modified  to  reduce  the  noise.  The  steel  wheels  running  in  the 
chaimels  were  replaced  with  simple  flat  plastic  pads  (made  from  high  density  polyethylene).  It 
was  much  quieter  in  operation  and  potentially  lighter  weight.  We  were  concerned  about 
potential  wear  and  monitored  it  as  testing  continued  -  the  plastic  surface  showed  some  galling 
after  operation.  Pads  were  required  facing  each  of  the  three  sides  of  the  channel  to  keep  the 
slider  from  twisting  under  high  accelerations.  There  was  more  friction  with  the  plastic  than 
with  the  wheels.  Spraying  the  inside  of  the  aluminum  channels  with  silicone  lubricant  reduced 
the  friction  considerably,  but  it  had  to  be  sprayed  fairly  often  -  something  like  once  a  day. 

Tuning  of  the  servo  loop  control  software  was  best  done  with  a  payload  which  moved 
smoothly  with  uniform  resistance  to  motion.  Since  the  temporary  slide  mechanism  did  not 
provide  smooth  motion,  the  payload  was  outfitted  with  a  wheeled  carriage  mechanism  which 
was  constructed  using  nylon  ball  bearing  wheels  to  which  rubber  O-ring  tires  were  attached 
[Figure  4.1-4].  While  it  provided  the  desired  smooth  (and  quiet)  operation  of  the  desktop 
prototype,  it  was  too  bulky  for  use  in  the  proof-of-concept  demonstrator. 
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Figure  4.1-4  Portion  of  the  payload  and  wheeled  slider  assembly  -  rubber  "tires"  on  the  wheels 

grip  a  flange  to  provide  quiet  operation. 

The  desktop  prototype  was  later  retrofitted  with  PTFE  slider  material  (a  high-performance 
plastic  material).  It  was  be  used  in  conjunction  with  stainless  steel  rails  which  provided 
smoother  motion  than  the  aluminum  rails  used  with  the  first  set  of  sliders.  The  galling  we 
witnessed  with  the  first  sliders  was  also  eliminated. 

Note  that  performance  of  the  original  steel  wheeled  arrangement  was  fine  except  for  the 
noise  generated.  Noise  is  mainly  a  cosmetic  issue.  Nonetheless,  using  plastic  sliders  had  the 
additional  benefits  of  being  lighter  and  simpler  than  either  of  the  wheeled  arrangements,  as 
well  as  generating  less  noise. 

We  noted  we  had  to  tighten  the  Kevlar'^'^  drive  cords  periodically.  While  not  a  major 
problem,  we  looked  into  the  cord  stretch  problem  by  conducting  some  tests.  We  determined 
that  when  in  constant  use,  the  cords  stretched  slightly  due  to  heating.  The  result  was  lower 
than  desired  cord  tension.  We  also  determined  that  as  Ihe  cords  cooled,  they  returned  to  proper 
tension.  We  do  not  believe  this  condition  will  exist  when  used  in  a  true  simulation  environment 
where  the  TOPIT  is  exercised  much  less  frequently  than  in  our  testing  scenario.  We  therefore 
decided  not  to  take  any  additional  action  other  than  to  continue  to  monitor  the  situation. 

4. 1.2  Manipulator  design 

We  define  the  manipulator  as  including  the  X,  Y,  and  Z  drives  and  related  hardware.  We 
designed  the  TOPIT  manipulator  based  upon  lessons  learned  with  the  desktop  prototype.  We 
decided  to  stay  with  the  cable-driven  mechanism  used  for  the  desktop  prototype,  rather  than 
switching  to  the  originally-proposed  belt-drive  arrangement.  The  cable  (or  "string")  drive 
worked  fine  on  the  desktop  prototype  and  has  the  advantage  of  keeping  the  servo  motor 
further  away  from  the  tracker.  We  also  decided  to  use  cable  drive  for  the  y-axis,  i.e.,  the  vertical 
axis  [Figure  4.1-5].  This  saved  the  x-axis  motor  from  having  to  move  a  y-axis  motor.  The  y-axis 
motor  remains  stationary.  The  y-axis  motor  is  coupled  to  the  Y  slider  through  pulleys. 


26 


Electric  motor 


Figure  4.1-5  How  a  stationary  motor  drives  the  y-axis  independent  ofx-axis  motion. 


We  combined  the  operator  station  with  the  manipulator  frame  design  to  reduce  the  overall 
complexity.  By  adding  brackets  to  the  manipulator  frame  to  hold  the  operator  seat  which  in 
turn  held  the  operator  hand  controls  (joystick  and  throttle)  [Figures  4.1-6  and  4.1-7]  we 
eliminated  virtually  aU  of  the  mechanical  design  effort  from  the  operator  station.  Since  the 
position  of  the  operator's  seat  and  hand  controls  are  fixed  to  the  manipulator  frame,  they  do 
not  have  to  be  tracked  during  real  time  simulation.  Note  the  hybrid  tracker  attached  to  the 
dataglove  [Figure  4.1-6]. 


Figure  4.1-6  Control  station  with  throttle  and  joystick;  HMD  rests  on  seat. 
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Figure  4.1-7  Control  station  includes  throttle  and  joystick;  HMD  rests  on  seat. 

Note  the  large  emergenq^  stop  switch  to  the  immediate  rear  of  the  user's  throttle  housing 
on  the  left  side  of  the  chair  [Figure  4.1-7].  The  HMD  rests  on  the  chair.  The  dataglove  is  in  a 
protective  box  without  the  hybrid  tracker  attached. 

We  decided  to  make  the  manipulator  with  two  side  A-frames  connected  by  horizontal  rails. 
All  manipulator  mechanical  components  were  attached  to  the  frame.  We  used  two-inch-square 
stainless  steel  tubing  for  the  frame  since  stainless  steel  causes  much  less  interference  with  the 
magnetic  tracker  than  carbon  steel  or  aluminum.  Wood  was  considered  briefly  but  then 
discoimted  since  wood  is  unstable  with  changes  in  humidity  and  it  is  difficult  to  produce  a 
sufficiently  stiff  structure  with  wood.  Stainless  steel  channels  were  attached  to  the  frame  to 
support  moving  parts  of  the  manipulator.  Large  X  and  Y-axis  drive  motors  are  located  at  the 
rear  of  the  manipulator  frame  to  minimize  magnetic  interference  with  the  user's  tracked  hand 
[Figures  4.1-8  and  4.1-9].  Note  the  drive  drums  and  Kevlar  drive  cords  in  both  figures. 
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Figure  4.1-8  Lower  right  side  of  manipulator  frame  showing  x-axis  drive  components. 


Figure  4.1-9  Lower  left  side  of  manipulator  frame  showing  y-axis  drive  components. 


From  the  drive  drums  shown  above  the  Kevlar  drive  cords  are  routed  via  pulleys  to  the 
payload  [Figure  4.1-10].  Cord  tension  is  provided  with  tumbuckles  such  as  the  one  shown  in 
the  figure. 
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Figure  4.1-10  Manipulator  frame  showing  Kevlar  cord,  tensioning  tumbuckle,  and  pulleys. 

From  the  pulleys  the  Kevlar  cord  is  routed  to  the  X  traveler  (tall  vertical  frame)  and  the  Y 
traveler  (behind  the  payload)  [Figure  4.1-11].  The  X  traveler  moves  horizontally,  supported  by 
the  manipulator  frame  at  the  top  and  bottom.  The  Y  traveler  moves  vertically,  supported  by  the 
vertical  side  channels  of  the  x-traveler.  Graphite  composite  stiffeners  reinforce  the  x-traveler. 


Figure  4.1-11  X-traveler,  y-traveler,  and  payload. 

A  ball  screw  drive  was  adopted  for  the  z-axis.  Linear  bearings  are  preferred  in  applications 
like  this  since  they  provide  smooth  linear  motion  but  such  bearings  weigh  significantly  more 
and  are  physically  larger  than  brass  bearings.  To  save  weight  and  space  brass  bearings  were 


used.  Unfortunately,  the  higher  friction  produced  by  the  brass  bearings  in  combination  with  an 
under-powered  (but  light  weight),  z-axis  drive  motor  did  not  work  well.  Motion  was 
intermittent  and  the  motor  often  stalled.  With  the  test  experience  we  now  have,  we  believe  the 
X  and  Y  drives  have  sufficient  power  to  permit  the  use  of  heavier  linear  bearings  and  a  larger 
z-axis  drive  for  any  follow-on  units  we  might  build. 

System  testing  with  high  accelerations  showed  imdesirable  X-axis  deflections  in  the 
manipulator  frame  caused  by  high  acceleration  moves  in  the  X  direction.  To  cure  this  problem, 
we  designed  a  set  of  dampers  for  the  marupulator  frame  to  reduce  the  vibration.  These 
dampers  are  currently  under  construction  and  will  be  installed  as  soon  as  construction  is 
complete. 

4. 1.3  Payload  design 

We  define  the  payload  as  including  the  touchboard  (the  panel  on  which  all  TOPIT  simulated 
cockpit  controls  are  mounted)  and  all  associated  servos,  solenoids,  and  other  hardware.  It  is  the 
payload  which  is  moved  by  the  manipulator. 

Rotary  and  toggle  switches  on  the  payload  had  to  be  rotated  so  that  the  user  would  find  the 
switch  in  the  correct  position  corresponding  to  the  image  of  the  virtual  switch  in  his  HMD.  A 
number  of  alternative  configurations  of  gears  and  motors  that  could  accomplish  the  needed 
rotation  were  considered.  The  simplest  way  would  have  been  to  directly  rotate  each  switch, 
without  gearing,  [Figure  4.1-12].  In  the  figure  solenoids  are  used  to  engage  switch  detentes 
imder  computer  control  so  that  switches  having  different  detente  spacing  could  be  simulated. 


Figure  4.1-12  Direct  drive  switch  rotation  with  programmable  detente  positions. 
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Working  from  the  sizes  and  weights  of  the  various  switches  and  the  properties  of  motors, 
the  design  implications  of  the  alternatives  were  studied.  Table  4.1-1  shows  an  example  of  the 
calculations  we  did  for  a  particular  motor  and  gear  configuration. 


Table  4.1-1  Payload  design  calculations. 


Switch 

Average 

Switch 

Speed 

(rad/sec) 

Switch 

acceleration 

(rad/sec^) 

Average 
motor 
speed 
(rad /sec) 

Motor 

acceleration 

(rad/sec^) 

Armature 
torque  (oz. 
in..) 

Friction 

torque 

(oz.in.) 

Load 

torque 

(oz.in.) 

Motor 

torque 

required 

(oz.in.) 

A1 

7,200 

10,048 

7,200 

10,048 

1.68 

0.17 

3.25 

5.10 

A2 

7,200 

10,048 

7,200 

10,048 

1.68 

0.17 

2.75 

4.59 

A3 

7,200 

10,048 

10,048 

1.68 

0.17 

0.55 

2.40 

A4 

7,200 

10,048 

7,200 

10,048 

1.68 

0.17 

0.37 

2.21 

B1 

5,400 

7,536 

5,400 

7,536 

1.96 

0.28 

7.63 

9.87 

B2 

5,400 

7,536 

5,400 

7,536 

1.96 

0.28 

7.63 

9.87 

B3 

5,400 

7,536 

5,400 

7,536 

1.96 

0.28 

7.63 

9.87 

B4 

5,400 

7,536 

5,400 

7,536 

1.96 

0.28 

7.63 

9.87 

The  rotary  switch  portion  of  the  payload  is  a  mechanism  that  moves  selector  switches  and 
continuous  rotary  controls  (like  volume  controls)  into  the  angular  positions  to  which  they  were 
last  set  by  the  user.  The  user  must  find  each  control  in  the  correct  angle  and  with  the  correct 
detentes  and  stops  for  that  control.  A  motor  turns  the  control  to  the  correct  position  and  a  set  of 
solenoids  selects  the  detentes.  A  second  motor  (not  shown)  selects  the  stop  angles  for  the 
selected  rotary  control.  The  stop  motor  controls  the  extreme  left  and  right  control  rotation 
angles. 

In  many  ways  the  rotary  control  portion  of  the  payload  design  is  the  most  demanding, 
because  a  complex  mechanism  must  be  put  in  a  small  space.  To  save  weight,  a  single  motor 
and  solenoid  mechanism  was  geared  to  drive  four  rotary  controls.  Only  one  of  the  four  rotary 
controls  can  be  accessed  at  any  one  time  by  the  user,  so  it  makes  no  difference  that  the  other 
knobs  linked  by  gears  happen  to  be  rotating  in  imison. 

4.1.4  Control  software  development 

At  the  beginning  of  the  project,  we  shared  some  of  the  assets  of  this  contract  with  the 
STRICOM  SBIR  A94-062  3-Axis  Locomotion  Simulator  Study  contract  for  basic  motion  control 
software  development  efforts.  Basic  motion  control  is  common  to  both  projects.  The  servo 
electronics  were  connected  to  a  servo  motor  we  installed  in  a  commercially  available  treadmill. 
The  treadmill  was  used  to  demonstrate  single  axis  control  for  the  SBIR  A94-062  project.  Control 
software  developed  for  the  treadmill  demonstrator  was  modified  for  use  on  the  TOPIT. 

Treadmill  demonstrator  control  software  accepted  tracker  position  data  which  indicated  the 
treadmill  user’s  position  on  the  treadmill  and  adjusted  the  speed  of  the  treadmill  to  prevent  the 
user  from  walking  or  running  off  either  end  of  the  treadmill.  User  tracking  was  first  done 
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mechanically  with  a  "stick"  tracker  -  a  piece  of  wood  with  one  end  attached  to  an  encoder  and 
the  other  end  held  by  the  treadmill  user.  The  stick  tracker  was  sufficient  for  its  purpose  and 
served  us  well  during  our  initial  motion  control  experiments. 

Efforts  then  progressed  to  the  Ascension  magnetic  tracker  later  in  the  effort.  Software  which 
phase  locks  the  PMAC  operations  to  those  of  the  PC  were  also  developed  and  proven  with  the 
treadmill  demonstrator.  This  phase  locking  software  was  directly  applicable  to  the  TOPIT. 

Safety  software  was  added  limit  the  speed  and  acceleration  of  the  servo  motor.  The  safety 
software  serves  two  purposes.  It  limits  ^e  speeds  and  accelerations  to  avoid  damaging  the 
servo  and  drive  mechanism,  and  it  protects  the  user  from  high  accelerations  that  might  cause  a 
loss  of  balance.  Initially,  the  limits  were  set  low  to  protect  the  user.  As  we  got  the  tracking  and 
control  algorithms  perfected,  the  envelope  of  the  treadmill  performance  was  expanded  to 
accommodate  more  vigorous  acceleration,  running,  and  stopping. 

Initial  motion  control  testing  on  the  desktop  prototype  was  done  with  user  input  directly  to 
the  PMAC  via  an  encoder  [Figure  4.1-13]  .  The  encoder  (upper  left  in  the  figure)  mounted 
temporarily  in  the  electronics  cabinet  was  used  to  provide  a  temporary,  electrically  noise-free 
source  of  hand  position  tracking.  Next  we  drove  an  analog  input  to  the  PC  with  a 
potentiometer  to  dieck  the  PC/PMAC  interface.  We  then  drove  the  system  with  the  magnetic 
tracker  input  to  the  PC.  Subsequent  testing  used  the  Multi-Sensor  Hybrid  Tracker  (discussed 
below).  This  gave  us  the  ability  to  move  the  tracker  and  have  the  desktop  demonstrator 
payload  follow.  This  step-by-step  checkout  procedure  allowed  us  to  thoroughly  check  each 
component  before  adding  another  uncertainty  to  the  system. 


Figure  4.1-13  Portion  of  the  electronics  and  the  encoder  used  for  initial  "string  tracking"  tests 
-  the  string  loop  extends  to  the  prototype  fixture  and  slider  motion  tracks  the  string  position. 

To  get  best  performance  from  the  system  it  was  necessary  to  tune  motor  performance  to  the 
actual  masses  and  spring  constants  of  the  respective  motor  loads.  Delta  Tau,  manufacturer  of 


33 


the  PMAC  motion  control  card  we  are  using,  provides  a  set  of  software  utilities  for  this 
purpose. 

4.2  Tracking  System 

Project  Objective  #2:  Ensure  the  tracking  system  provided  sufficient  accuracy  in  the  presence  of 
electromagnetic  noise  and  moving  metal  objects. 


Our  hybrid  tracker  consists  of  a  commercially  available  Ascension  Flock  of  Birds  magnetic 
tracker  combined  with  a  custom  six  degree-of-freedom  inertial  tracker.  The  magnetic  tracker 
produces  both  position  and  attitude  data  but,  to  minimize  inherent  noise,  data  averaging  of 
many  sequential  data  points  is  used.  This  averaging  process  introduces  a  time  lag  in  the 
position  and  attitude  data  sent  to  the  computer.  A  second  problem  with  magnetic  trackers  is 
their  susceptibility  to  interference  by  metallic  objects  -  particularly  those  containing  aluminum, 
steel,  or  iron. 

To  solve  the  lag  problem  we  constructed  a  hybrid  tracker  by  adding  inertial  tracking  to  the 
magnetic  tracker.  The  inertial  tracker  measures  X,  Y,  and  Z  linear  accelerations  as  well  as 
rotational  accelerations  in  roll,  pitch,  and  yaw.  Kalman  filter  software  running  in  a  Pentium  Pro 
200  is  used  to  combine  the  data  from  the  magnetic  and  inertial  sensors  to  provide  accurate,  low 
lag  hand  position  and  attitude  information  for  the  simulation. 

4.2.1  Hybrid  tracker  development 

We  ran  some  preliminary  experiments  to  determine  the  tracker's  susceptibility  to  interference 
from  non-moving  steel  and  aluminum  objects.  The  tracker  was  also  placed  near  a  2  horsepower 
electric  motor.  The  motor  was  turned  on  and  off  to  roughly  simulate  what  might  happen  with  a 
servo.  The  results  showed  about  what  we  expected:  the  tracker  is  quite  susceptible  to  the  motor 
noise.  We  also  ran  some  additional  experiments  to  determine  the  susceptibility  to  interference 
from  non-moving  carbon  steel,  stainless  steel,  and  aluminum  objects  which  were  placed 
nearby.  There  were  several  interesting  results.  First,  the  tracker  exhibits  a  large  error  at  longer 
tracker  ranges  (24  inches  or  more)  and  the  error  is  not  affected  much  by  nearby  non-moving 
interfering  test  objects.  We  also  noted  that  stainless  steel  test  objects  had  no  noticeable  affect  on 
the  tracker  accuracy.  We  also  determined  that  the  shape  of  aluminum  test  objects  had  a  large 
effect  on  the  tracker  accuracy. 

A  detailed  list  of  tests  and  software  required  to  test  the  magnetic  tracker  was  prepared.  The 
purpose  of  those  tests  was  to  determine  the  effect  of  stainless  steel,  carbon  steel,  and  aluminum 
on  the  accuracy  of  the  tracked  position.  For  this  series  of  tests,  test  objects  were  placed  at 
various  distances  from  the  tracker  receiver  and  the  tracker  receiver  will  be  placed  at  various 
distances  from  the  tracker  transmitter.  Test  objects  included  one  inch  round  by  two  foot  long 
tubes  and  one  foot  by  two  foot  sheets  of  the  three  metals  mentioned  above.  Test  results  showed 
no  significant  interference  with  series  300  stainless  steel  while  also  showing  there  was 
substantial  interference  with  both  carbon  steel  and  aluminum  and  thus  confirmed  what  we  had 
been  told  by  the  magnetic  tracker  manufacturer  Ascension. 

We  looked  at  ways  to  overcome  the  noise  susceptibility  of  the  magnetic  tracker,  and 
developed  an  approach  which  used  inertial  sensors  in  conjunction  with  the  magnetic  tracker. 
Inertial  sensors,  miniature  accelerometers  and  angular  rate  sensors,  provide  excellent  short¬ 
term  accuracy  that  is  immune  to  electromagnetic  effects.  However,  the  inertial  measurements 
drift  over  time,  and  must  be  updated  with  position  and  angle  "fixes"  to  remove  the  drift  errors. 
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If  magnetic  tracker  measurements  are  available  occasionally  to  update  the  inertial 
measurements,  then  we  expected  the  tracking  system  would  perform  well  over  both  the  short 
term  and  long  term. 

Kalman  Filter  Software 


The  Kalman  filter  software  combines  the  data  from  all  the  sensors  under  consideration  to 
provide  a  combined  "optimal"  estimates  of  position  and  attitude.  The  Kalman  filter  software 
actually  combines  data  from  various  sensors  continually,  weighting  the  value  of  each  data 
according  to  its  error  characteristics.  It  works  amazingly  well.  Kalman  filter  technology  has 
been  used  in  aerospace  applications  such  as  aircraft  navigation  for  a  long  time. 

Initially,  most  of  the  sensor  error  models  were  based  on  published  specifications,  but  as  we 
collected  data  the  error  models  were  improved. 

Kalman  filter  software  requires  "tuning"  as  part  of  the  test  process  before  it  will  function 
properly.  Tuning  is  the  adjustment  of  the  mathematical  models  of  the  sensor  error  performance 
to  agree  with  the  error  performance  encoimtered  in  practice.  Paul  worked  with  one  of  our 
student  interns  to  time  the  software. 

In  the  development  we  captured  a  set  of  hybrid  tracker  data  for  a  small  set  of  hand 
motions:  left  -  right,  fore  -  aft,  and  a  series  of  rolls.  The  data  were  used  to  time  the  Kalman  filter 
software.  Rather  than  sampling  data  at  different  times,  all  of  our  data  is  sampled  at  one  instant 
in  time.  Changes  were  made  to  the  software,  which  was  originally  designed  to  sample  data  at 
different  times,  to  accommodate  the  new  data  sampling  scheme  and  forwarded  revised 
software  to  CGSD. 

Magnetic  field  calibration  software 

Despite  the  use  of  series  300  stainless  steel  for  most  custom  portions  of  the  TOPIT  the  magnetic 
field  of  the  tracker  was  still  distorted  by  the  presence  of  solenoids,  motor,  switches  and  other 
non-stainless  steel  metallic  items.  To  get  the  needed  accuracy  from  the  magnetic  tracker  in  the 
presence  of  these  items  we  developed  software  to  map  the  position  dependent  magnetic  field 
distortions  for  the  magnetic  tracker.  The  magnetic  tracker  was  attached  to  the  payload.  The 
payload  was  driven  to  various  positions  by  custom  calibration  software.  The  position  and 
attitude  at  each  grid  location  was  recorded  in  a  table.  The  contents  of  the  magnetic  calibration 
table  were  subsequently  used  by  the  real  time  software  to  determine  the  actual  location  of  the 
magnetic  tracker. 


TEU  Hardware 


At  the  time  we  started  development  of  the  hybrid  tracker  we  had  three  contracts  which 
required  advanced  tracker  technology;  this  contract,  the  STRICOM  SBIR  A94-062  3-Axis 
Locomotion  Simulator  Study  contract,  and  a  commercial  contract.  We  constructed  a  tracker 
evaluation  unit  (TEU)  which  includes  X,  Y,  and  Z  accelerometers,  roll,  pitch,  and  yaw  angular 
rate  sensors,  a  tilt  sensor,  and  a  compass.  This  module  along  with  custom  software  was  used  to 
select  the  sensor  configuration  most  appropriate  for  each  of  the  three  applications.  A  printed 
circuit  board  was  designed,  fabricated,  assembled  [Figure  4.2-1],  and  tested.  As  described 
above,  the  TEU  has  a  large  collection  of  sensors  and  is  designed  strictly  to  support  lab 
experimentation.  It  is  too  large  and  has  too  many  sensors  for  production.  However,  it  is 
essential  for  collecting  the  real  data  we  need  to  support  algorithm  development. 
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Figure  4.2-1  The  Tracker  Evaluation  Unit  contains  more  sensors  than  were  ultimately 

selected. 

Hybrid  Tracker  2  Hardware 

We  decided  to  repackage  the  hybrid  tracker  so  that  it  would  not  be  as  bulky  and  heavy.  The 
original  TEU  discussed  above  contains  a  compass  and  tilt  sensor  which  are  not  required  by  the 
TOPIT  program.  The  redesigned  unit,  called  "hybrid  tracker  2",  consists  of  two  modules;  a 
sensor  module  shown  with  toe  magnetic  tracker  [Figure  4.2-2]  which  contains  the  gyros  and 
accelerometers  and  the  control  module  for  the  computer  interface.  The  custom  printed  circuit 
board,  compass,  and  tilt  sensors  used  in  toe  TEU  were  eliminated.  The  sensor  module,  which  is 
attached  to  the  back  of  the  glove,  is  much  smaller  and  lighter  than  TEU.  It  is  connected  to  the 
control  module  by  a  single  cable. 
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Figure  4.2-2  Hybrid  tracker  2  sensor  module  with  magnetic  tracker  attached. 


We  believe  a  production  version  of  the  hybrid  tracker  can  be  considerably  smaller  and 
lighter  weight  than  hybrid  tracker  2  shown. 

Hybrid  Tracker  3  Hardware 

Our  research  showed  that  the  alignment  of  the  sensors  in  the  Hybrid  Tracker  2  unit  was  not 
acceptable.  As  a  result  we  integrated  two  triaxial  sensors  -  a  rate  gyro  sensor  unit  and  an 
accelerometer  sensor  unit.  This  replaced  the  inertial  tracking  portion  of  the  Hybrid  Tracker  2 
unit.  We  are  still  using  the  Flock  of  Birds  magnetic  tracker  unit.  Hybrid  Tracker  3  is  lighter  and 
has  a  smaller  footprint  than  its  precursors.  Signal  conditioning  hardware,  built  by  one  of  our 
student  interns,  was  used  to  integrate  the  new  sensors  with  the  existing  system. 


4.2.2  Optical  Tracker  Development 

Since  the  result  of  the  hybrid  tracker  is  not  quite  up  to  our  expectations,  we  are  further 
exploring  the  optical  tracker. 

We  have  installed  DynaSight™  sensor  (optical  tracker  by  Origin),  have  written  drivers  for 
the  optical  tracker,  and  integrated  the  optical  tracker  into  TOPIT. 

Two  methods  were  integrated  in  order  to  test  the  applicability  of  each.  The  two  methods 
are  described  below: 


Method  1: 


Method  one  uses  a  single  optical  target.  The  target  used  was  initially  just  a  piece  of  retro- 
reflective  tape  applied  to  the  glove's  index  finger.  Offsets  were  recalculated  for  this  tracking 
system  and  the  hand  position  was  backwards  calculated  from  the  glove  offsets  in  order  to 
simulate  the  finger  articulations.  The  hand  orientation  was  read  in  from  the  magnetic  tracker. 

This  proved  to  get  rid  of  some  of  the  position  noise  as  well  as  problems  with  the  warped 
magnetic  field  (caused  by  the  stainless  steel  frame  and  the  EM  fields  from  the  motors  and 
solenoids  in  the  payload);  however,  after  testing,  a  position  hysteresis  was  found  while  moving 
along  the  X  axis  of  the  optical  tracker.  A  constant  offset  was  noticed  with  constant  velocity. 
This  was  a  result  of  the  sequential  (180  degrees  our  of  phase)  measurements  taken  by  each 
optical  sensor  as  it  calculated  the  Z  axis. 

The  retro-reflective  tape  was  exchanged  for  an  active  target  (IR  LED).  Integrating  the  IR 
LED,  the  ATA  (Active  Target  Adapter),  and  the  TOPIT™  system  was  easily  accomplished.  DIP 
switches  in  the  back  side  of  the  optical  tracking  unit  can  be  set  to  use  the  ATA  and  IR  LED 
(please  refer  to  the  DynaSight^^  Sensor  manuals). 

Method  2: 

Method  two  uses  three  optical,  active  targets  in  order  to  calculate  the  hand  orientation  as  well 
as  position.  This  required  writing  a  driver  for  the  QNX  operating  system.  The  ATA  integrated 
easily  with  the  DjmaSight™  Sensor  (DIP  switches  must  be  adjusted.  Please  refer  to  the 
DynaSighf^^  Sensor  manuals).  The  three  active  targets  were  moxmted  on  a  triangular  plate 
(supplied  by  Origin  Instruments)  and  moimted  on  the  CyberGlove. 

The  purpose  of  using  optical  sensors  was  to  increase  the  position  accuracy  of  the  sensors. 
Without  filtering,  the  position  measurements  were  much  less  noisy  compared  to  the  magnetic 
trackers.  Also,  since  the  optical  trackers  are  not  affected  by  EM  waves,  the  warped  field  did  not 
affect  its  accuracy. 

Notes  in  comparing  the  noise  to  that  of  the  magnetic  tracking  unit  (MT)  are  as  follows: 

Noise  reading:  (taken  on  26  February  1998) 

MT  with  ALL  filters  on  (lots  of  lag  -  unacceptable  lag) 
position:  0.02  inches 
angle:  0.03  degrees 

MT  with  ACNarrow  filter  only  (normal  operation  mode) 
lag  is  okay... 
position:  0.85  inches 
angle:  1.70  degrees 
OPTICAL  (no  filtering) 
lag  is  okay. . . 
position:  0.06  inches 
angle:  1.45  degrees 

Because  the  magnetic  tracker  measurements  with  all  of  the  filters  on  creates  an  unacceptable 
lag,  these  noise  figures  are  not  used  as  comparison.  With  the  ACNarrow  filters  on  only,  the 
magnetic  tracker  data  contains  more  noise  compared  to  that  of  the  optical  tracker. 

Method  1  with  an  active  target  (IR  LED)  is  used  even  though  its  implementation  does  NOT 
output  angle  information.  The  reason  for  this  is  that  the  position  of  the  fingertip  is  given  rather 
than  the  position  of  the  wrist  (as  the  case  would  be  in  Method  2).  If  the  sensor  is  used  to 
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calculate  the  wrist  position,  the  angle  noise  would  affect  the  calculated  fingertip  position. 
Therefore,  Method  1  provides  a  more  accurate  solution  to  the  fingertip  tracking  problem. 

4.3  Hand  Motion  Prediction  Algorithms 


Project  Objective  #3:  Design  hand  motion  prediction  algorithms  that  predict  which  control 
will  be  touched  while  sufficient  time  remains  to  put  it  in  place. 


Software  to  meet  this  objective  consists  of  two  parts;  the  hand  motion  prediction  algorithm 
itself,  and  the  cockpit  to  real  time  capture  zone  software. 

Hand  motion  prediction 

The  hand  motion  prediction  algorithm  was  implemented  by  starting  with  current  hand 
position  and  attitude.  Predicted  positions  and  attitude  is  determined  by  using  accelerations  and 
velocities  for  six  degrees  of  freedom.  The  general  form  of  the  computation,  which  is  performed 
in  all  six  axes  of  motion  (X,  Y,  Z,  roll,  pitch,  and  yaw),  is  as  shown  in  equation  1.  The  hand 
prediction  algorithm  is  working  well. 


Pp  =  Pc  +  VcT  +  AcT2 


where: 

Pp  =  predicted  position 
Pc  =  current  position 
Vc  ==  current  velocity 
Ac  =  current  acceleration 
T  =  time  to  predicted  position 


Zone  capture  software 

It  was  useful  to  divide  the  operating  envelope  of  the  virtual  cockpit  into  regions  or  "zones"  that 
help  define  TOPIT  FTPS  manipulator  operating  conditions  and  operating  actions.  The  four 
zones  are: 

Zone  A:  A  volume  ordinarily  containing  the  user,  when  the  user  is  not  accessing  any 
controls  on  the  virtual  control  panel. 

Zone  B:  A  volume  outside  Zone  A  extending  to  within  a  short  distance  -  about  1.5 
inches  -  of  the  surfaces  of  the  front  of  the  virtual  control  panel  with  its 
instruments. 

Zone  C:  The  volume  on  the  user  side  of  the  virtual  instruments  within  a  short 

distance  -  about  1.5  inches  -  of  the  virtual  control  panel  with  its  instruments 
but  outside  Zone  B. 
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Zone  D:  The  volume  beyond  Zone  C,  including  the  surfaces  of  the  controls  and  the 
control  panel. 


Figure  4.3-1  illustrates  a  cross-section  with  the  various  manipulator  control  zones. 
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Figure  4.3-1  Manipulator  Control  Zones. 


Note  that  for  the  prototype  TOPIT,  the  controls  need  not  lie  in  a  plane.  The  transport 
includes  a  depth  axis  which  allows  the  virtual  control  panel  to  be  stepped  or  even  curved. 
Accordingly,  the  control  zone  volumes  A-D  are  mapped  to  the  contours  of  the  cockpit  control 
panel  in  a  lookup  table  and  are  not  bounded  by  planes. 

The  requirements  for  TOPIT  FTPS  positioning  speed  depend  upon  the  position  of  the  user's 
hand  relative  to  the  operating  zones  described  above.  Motion  requirements  for  each  operating 
zone  follow: 


Zone  A:  When  the  user's  tracked  hand  is  within  Zone  A,  the  TOPIT  FTPS  will  not  move. 

Zone  B:  When  the  user's  tracked  hand  is  in  Zone  B,  the  TOPIT  FTPS  will  move  at 
maximum  speed  to  the  mirror  point.  The  mirror  point  is  that  manipulator 
position  which  is  closest  to  the  user's  tracked  hand. 

Zone  C:  When  the  user's  tracked  hand  is  in  Zone  C,  the  TOPIT  FTPS  marupulator  will 
move  at  maximum  speed.  In  this  zone  the  computer  determines  which  control 
the  user  is  reaching  for  and  positions  the  appropriate  control  at  the  correct 
position  in  front  of  the  user.  The  choice  made  by  the  computer  will  be  based 
upon  extrapolation  of  the  hand  trajectory.  The  final  position  of  the  TOPIT  FTPS 
is  not  made  until  the  user's  hand  enters  zone  D. 
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Zone  D:  When  the  user's  tracked  hand  is  in  Zone  D,  the  TOPIT  FTPS  will  not  move. 


The  real  time  capture  zone  software  that  provides  these  functions  operates  properly. 

Early  in  debug  we  discovered  the  need  for  an  additional  refinement.  As  originally 
implemented,  the  control  software  would  always  attempt  to  use  the  maximum  performance  of 
the  hardware  to  track  the  hand.  It  would  accelerate  the  payload  at  4  Gs  to  follow  the  hand 
motion,  even  when  the  hand  was  relatively  far  away  from  the  simulated  panel  surface  in  zone 
B.  The  resulting  violent  motion  of  the  hardware  place  unnecessary  stress  on  the  mechanism, 
because  high  accelerations  are  only  required  when  the  users  hand  is  relatively  near  the  panel. 

To  damp  the  motion  of  the  payload  when  the  hand  is  distant,  we  implemented  a  variable 
time  constant  filter  that  smoothes  the  motion  tracking.  When  the  hand  is  far  away,  the  time 
constant  is  large  and  high  accelerations  are  avoided.  The  time  constants  are  reduced  to  provide 
full  performance  as  the  hand  approaches  a  control  on  the  panel. 

Note  that  the  additional  smoothing  is  not  applied  to  the  position  used  for  rendering  the 
image  of  the  hand.  The  image  always  react  quickly  so  the  user  will  have  a  correct  view  of  his 
hand  position.  The  extra  filtering  is  not  applied  to  the  extrapolated  hand  position  used  to  select 
which  control  will  be  actuated.  The  extra  filtering  is  only  applied  to  the  payload  positioning 
commands. 

4.4  Computation  Control  Lags 


Project  Objective  #4:  Keep  computation  and  control  lags  small  enough  so  that  the  positioning 
system  had  sufficient  time  to  position  the  touch-board. 


The  following  steps  were  taken  to  minimize  computation  and  control  lags  in  the  system: 


•  Dedicated  servo  controllers  were  used  to  provide  high  computational  rates  to  support  the 
servo  loop  computations.  The  loop  update  rates  are  over  lOOHz. 

•  True  real  time  operating  systems  were  used  throughout:  Ultrix,  SGI's  real  time  version  of 
UNIX  in  the  Onyx;  QI^,  a  proprietary  real  time  operating  system,  in  the  PC;  and  the 
PMAC  motion  control  system  in  the  servo  controllers.  True  real  time  systems  allow  the 
user  to  manage  the  priorities  of  interrupts  so  that  time  critical  events  are  not  delayed  in  a 
service  queue. 

•  A  dedicated  EthemeF'^  link  was  used  between  the  PC  and  the  On5oc.  Having  a  dedicated 
link  avoids  latency  due  to  packet  collisions,  and  latency  is  kept  under  one  millisecond. 

•  The  hybrid  tracker  provides  accurate  rate  and  acceleration  data  to  extrapolate  over 
unavoidable  latencies,  such  as  the  time  in  the  image  generator  needed  to  render  the 
graphics  imagery. 

Overall,  system  latencies  are  quite  good,  but  there  are  two  limitations.  First,  we  could  not 
afford  to  build  a  second  hybrid  tracker  for  the  head  mounted  display,  so  there  are  noticeable 
lags  when  head  motion  is  rapid.  There  is  no  technical  problem  in  adding  a  second  hybrid 
tracker.  Second,  even  though  a  substantial  amoimt  of  the  budget  was  devoted  to  obtaining  a 
high  performance  image  generator,  and  considerable  effort  was  made  to  minimize  polygon 
coxmts  in  the  database,  the  image  generator  frame  rate  is  at  most  30Hz,  and  it  sometimes  drops 
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as  low  as  20Hz.  Image  generator  technology  has  advanced  rapidly,  so  that  now  at  the  end  of 
the  two  year  development  program  there  are  available  image  generators  with  much  higher 
polygon  capacities  than  the  SGI  RealityEngine2.  For  example,  the  newer  Lockheed-Martin 
RealSD  Pro  2000  provides  about  three  times  the  polygon  capacity  at  one-third  the  cost  of  the 
RealityEngine2. 


4.5  Safety  Systems 


Project  Objective  #5:  Provide  redundant  safety  systems  to  protect  the  operator  during 
development  and  use. 

A  number  of  things  were  done  to  ensure  personnel  and  equipment  safety.  Some  of  these 
features  were  discussed  briefly  in  previous  sections.  These  design  features,  combined  with  a 
few  common  sense  procedures  have  served  us  well  -  we  have  had  no  injuries  on  the  TOPIT 
program.  The  design  features  and  procedures  are  discussed  below. 

To  ensure  the  user's  head  is  not  hit  by  the  moving  payload,  should  he  lean  forward  for 
some  reason,  the  user's  seat  is  positioned  as  far  away  from  the  manipulator  frame  as  practical. 
The  seat  position  still  allows  the  user  to  activate  touchboard  controls  without  excessive  leaning. 

To  ensure  the  user’s  untracked  left  hand  is  never  near  the  moving  mechanism,  the  user 
must  depress  a  button  on  the  throttle  with  his  left  thumb.  If  he  lets  go  of  the  button,  real-time 
software  causes  X  and  Y  motions  to  stop  (other  motions  are  considered  not  dangerous).  The 
system  must  be  reinitialized  by  the  system  operator  before  it  will  move  again. 

Limit  switches  on  the  X,  Y  and  Z  axes  are  used  for  system  initialization  [Figure  4.5-1].  Limit 
switches  are  located  near  the  ends  of  the  excursions  on  each  axis  and  are  activated  if  the 
payload  touches  them.  Limit  switches  should  not  be  activated  during  real  time  simulation,  i.e., 
a  motion  control  error  has  occurred  if  they  do.  If  any  of  these  switches  are  activated  during  real 
time  simulation,  the  PMAC  motion  controller  software  causes  all  motions  to  stop.  The  system 
must  be  reinitialized  by  the  system  operator  before  it  will  move  again. 
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Figure  4.5-1  Typical  limit  and  emergency  limit  switches. 


A  set  of  emergency  limit  switches  is  located  next  to  but  beyond  the  X  and  Y  limit  switches 
[Figure  4.5-1].  Should  the  limit  switches  or  the  PMAC  software  discussed  above  fail  to  act  for 
any  reason,  the  emergency  limit  switches  will  cause  the  X  and  Y  drive  motor  windings  to  be 
disconnected  from  their  respective  amplifiers  and  shorted  together.  This  will  cause  motion  in 
both  the  X  and  Y  axes  to  stop  immediately.  Two  manually  operated  emergency  stop  switches, 
one  located  on  the  electronics  cabinet  and  one  near  the  user's  left  hand,  can  also  be  used  to  stop 
X  and  Y  motion.  Manual  reset  is  necessary  to  reactive  the  system  following  such  a  shut  down. 

A  potential  hazard  on  any  mechanical  design  is  sharp  edges.  These  were  mininuzed  during 
the  design  process.  With  a  moving  mechanism,  such  as  the  TOPIT,  the  concern  becomes  the 
existence  and  control  of  pinch  points.  We  use  both  bumpers  and  shields  (not  yet  installed)  to 
minimize  personnel  exposure  to  pinch  points. 

To  minimize  exposure  of  the  user's  tracked  right  hand  to  contact  with  moving  parts  of  the 
TOPIT  we  programmed  the  hand  motion  prediction  algorithms  to  stop  X,  Y,  and  Z  motions 
and  freeze  tide  position  of  the  payload  when  the  hand  approaches  the  payload.  Payload  motion 
does  not  resume  imtil  the  user  moves  his  hand  away  from  the  payload  again. 

A  safety  light  flashes  when  manipulator  power  is  on.  The  safety  light  is  positioned  so  that  it 
can  be  seen  by  anyone  in  the  area  of  the  TOPIT  (except  the  user  when  he  dons  the  HMD). 

Turning  to  procedures,  we  have  a  two  man  rule  for  TOPIT  operation  when  the  user  is 
wearing  the  HMD  and  cannot  see  the  manipulator.  The  second  man  positions  himself  close  to 
one  of  the  manually  operated  emergency  stop  switches  so  that  he  can  activate  the  switch 
should  he  see  anything  out  of  the  ordinary. 

To  prevent  a  passer-by  from  being  injured,  should  a  computer  glitch  cause  unexpected 
manipulator  motion,  we  insist  TOPIT  personnel  shut  off  manipulator  power  when  they  are  not 
in  the  vicinity  of  the  machine. 


5.  Conclusions 


Overall,  the  major  technical  challenges  were  met.  In  particular,  robotic  hardware  was  built  to 
position  the  controls  with  the  speed  and  accuracy  required,  and  a  sophisticated  tracker  and  an 
alternative  tracker  were  built  to  provide  the  accuracies  required  for  position  and  extrapolation. 
The  most  difficult  aspect  of  the  program  turned  out  to  be  getting  all  of  the  bugs  out  of  the 
complex  system  under  severe  budget  constraints.  In  this  last  respect  we  were  largely 
successful,  but  not  entirely.  The  main  limitations  of  the  final  prototype  lie  in  the  fine  points  of 
getting  the  software  to  run  completely  smoothly  and  reliably.  We  view  none  of  the  present 
limitation  as  being  fundamental. 
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